Add `WHEN STALE` syntax for `CREATE MATERIALIZED VIEW` #27356

piotrrzysko · 2025-11-18T09:27:40Z

Description

This PR introduces a new optional clause for CREATE MATERIALIZED VIEW that allows users to control the behavior of materialized views when they are stale.

I will updated the documentation once the proposed syntax and behavior are approved.

New syntax

CREATE MATERIALIZED VIEW mv_name
WHEN STALE { FAIL | INLINE }
AS SELECT ...

Behavior

FAIL - If the MV is stale, queries referencing it will fail. Analysis of the underlying query is never performed while querying the MV.
INLINE (default) - Preserves the current behavior. Even if the MV is stale, it is expanded like a logical view, and the underlying query is analyzed.

Motivation

Avoid analyzing the underlying query every time, which can be costly when it references many tables.
Allow using fresh MVs even when some data sources are temporarily unavailable.

Additional context and related issues

The proposed clause was originally discussed in Define grace/staleness period for materialized views #15326
Materialized view should only check permissions of underlying query for refresh #23747

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

piotrrzysko · 2025-11-18T09:40:05Z

core/trino-main/src/main/java/io/trino/sql/analyzer/StatementAnalyzer.java

                    TableHandle tableHandle = metadata.getTableHandle(session, storageTableName)
                            .orElseThrow(() -> semanticException(INVALID_VIEW, table, "Storage table '%s' does not exist", storageTableName));
-                    return createScopeForMaterializedView(table, name, scope, materializedViewDefinition, Optional.of(tableHandle));
+                    return createScopeForMaterializedView(table, name, scope, materializedViewDefinition, Optional.of(tableHandle), useLogicalViewSemantics);


I'm a bit concerned about this line: the new clause affects the path for fresh MVs (WHEN STALE FAIL skips analysis, while WHEN STALE INLINE runs the analysis), even though the syntax suggests it should only define behavior for stale MVs.

Is analysis of the underlying query even necessary for fresh MVs, regardless of the WHEN STALE clause? I understand that it can detect schema changes in the base tables or changes to the MV definer’s access permissions - but is that the intended behavior?

I understand that it can detect schema changes in the base tables or changes to the MV definer’s access permissions - but is that the intended behavior?

i think that was intended behavior...

Is analysis of the underlying query even necessary for fresh MVs, regardless of the WHEN STALE clause?

... however i think it should be skipped. this would be significant benefit when MVs are used in low latency scenarios.
in fact, for an MV with 24h grace period that was refreshed with last day, we should skip this line during a SELECT query, because it doesn't matter

trino/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/IcebergMetadata.java

Line 3906 in 0ce5929

tableChangeInfoTasks.add(() -> getTableChangeInfo(session, tableToSnapShot));

let's file an issue for this and follow-up separately

Praveen2112 · 2025-11-18T10:18:58Z

What is our plan on changing this behaviour for existing or new MVs ?

core/trino-main/src/main/java/io/trino/execution/CreateMaterializedViewTask.java

Praveen2112 · 2025-11-18T10:31:25Z

core/trino-main/src/main/java/io/trino/sql/analyzer/StatementAnalyzer.java

                }
                // This is a stale materialized view and should be expanded like a logical view
-                return createScopeForMaterializedView(table, name, scope, materializedViewDefinition, Optional.empty());
+                return createScopeForMaterializedView(table, name, scope, materializedViewDefinition, Optional.empty(), useLogicalViewSemantics);


Instead of passing a useLogicalViewSemantics can we create a scope for the storage table ?

Praveen2112 · 2025-11-18T10:39:08Z

Apart from the benefit on skipping the analysis what are the other benefits would I get if a run a query as WHEN STALE FAIL ? Now when I create a MV with WHEN STALE - I won't be able to use it unless I run a REFRESH MV. In case of pipeline which uses this MV might be affected if a MV went stale ? Is this expected ? Should we add this as an extension of WHEN FRESH SKIP ANALYSIS or something similar ?

martint · 2025-11-18T16:46:53Z

For context, #15326

martint · 2025-11-18T16:51:16Z

Also, #23387 (reply in thread) and #23747

piotrrzysko · 2025-11-19T10:40:01Z

@Praveen2112

What is our plan on changing this behaviour for existing or new MVs ?

This change is meant to be backward-compatible. The new clause is optional. If it is not specified, or if WHEN STALE INLINE is used, the MV will behave as it does currently.

Apart from the benefit on skipping the analysis what are the other benefits would I get if a run a query as WHEN STALE FAIL ? Now when I create a MV with WHEN STALE - I won't be able to use it unless I run a REFRESH MV. In case of pipeline which uses this MV might be affected if a MV went stale ? Is this expected ?

Skipping the analysis is not the goal in itself. The goal is to achieve better availability when the source tables are not available and to get closer to the semantics mentioned in the issues that Martin and I linked in this PR - specifically, that within the grace period, the MV should be treated as if it were a regular storage table. And today, as far as I know, the only blocker for this is the analysis performed every time the MV is queried.

Should we add this as an extension of WHEN FRESH SKIP ANALYSIS or something similar ?

This is more or less what I wanted to express in this comment: #27356 (comment). I’m happy to discuss what the syntax should look like to achieve the goals described above. Personally, WHEN FRESH SKIP ANALYSIS feels too low-level - as mentioned in my comment, I'd consider always skipping analysis when the MV is fresh.

piotrrzysko · 2025-11-26T13:54:21Z

@martint just a gentle reminder about the PR when you have a moment

findepi · 2025-11-26T16:26:45Z

@piotrrzysko can you please rebase? there is a conflict.

core/trino-main/src/main/java/io/trino/sql/rewrite/ShowQueriesRewrite.java

findepi

Add WHEN STALE FAIL/INLINE syntax

lgtm

findepi

Add when stale behavior in ConnectorMaterializedViewDefinition

core/trino-main/src/main/java/io/trino/execution/CreateMaterializedViewTask.java

core/trino-main/src/test/java/io/trino/sql/analyzer/TestAnalyzer.java

testing/trino-testing/src/main/java/io/trino/testing/BaseConnectorTest.java

findepi

please rebase to resolve a conflict

findepi · 2025-12-03T12:07:34Z

core/trino-main/src/main/java/io/trino/sql/analyzer/StatementAnalyzer.java

-            if (node.getWhenStaleBehavior().isPresent()) {
-                throw new TrinoException(NOT_SUPPORTED, "WHEN STALE is not supported yet");
-            }


Not sure whether Add when stale behavior in ConnectorMaterializedViewDefinition commit should be removing those lines.

This is needed to test whether MATERIALIZED_VIEW_WHEN_STALE_BEHAVIOR is respected, which happens during execution:

trino/core/trino-main/src/main/java/io/trino/execution/CreateMaterializedViewTask.java

Lines 163 to 165 in d9e7986

if (!plannerContext.getMetadata().getConnectorCapabilities(session, catalogHandle).contains(MATERIALIZED_VIEW_WHEN_STALE_BEHAVIOR)) {

throw semanticException(NOT_SUPPORTED, statement, "Catalog '%s' does not support WHEN STALE", catalogName);

}

One line below, I throw NOT_SUPPORTED even if a connector claims to support WHEN STALE:

trino/core/trino-main/src/main/java/io/trino/execution/CreateMaterializedViewTask.java

Line 166 in d9e7986

throw semanticException(NOT_SUPPORTED, statement, "WHEN STALE is not supported yet");

This is a preparatory change that allows connectors to store information about the WHEN STALE behavior. The option cannot yet be used in practice, as execution will still fail with a NOT_SUPPORTED error.

cla-bot bot added the cla-signed label Nov 18, 2025

github-actions bot added iceberg Iceberg connector lakehouse labels Nov 18, 2025

piotrrzysko added the syntax-needs-review label Nov 18, 2025

piotrrzysko commented Nov 18, 2025

View reviewed changes

piotrrzysko requested review from findepi, kasiafi and martint November 18, 2025 10:15

piotrrzysko marked this pull request as ready for review November 18, 2025 10:15

Praveen2112 reviewed Nov 18, 2025

View reviewed changes

findepi reviewed Nov 26, 2025

View reviewed changes

core/trino-main/src/main/java/io/trino/sql/rewrite/ShowQueriesRewrite.java Show resolved Hide resolved

findepi reviewed Nov 26, 2025

View reviewed changes

core/trino-main/src/main/java/io/trino/execution/CreateMaterializedViewTask.java Show resolved Hide resolved

findepi reviewed Nov 26, 2025

View reviewed changes

core/trino-main/src/test/java/io/trino/sql/analyzer/TestAnalyzer.java Show resolved Hide resolved

piotrrzysko mentioned this pull request Nov 27, 2025

Fix testAnalyzeMaterializedViewWithAccessControl #27477

Merged

piotrrzysko force-pushed the when_stale branch from da71e4a to 8d70622 Compare November 27, 2025 14:44

findepi changed the title ~~Add WHEN STALE option to CREATE MATERIALIZED VIEW~~ Add WHEN STALE option to CREATE MATERIALIZED VIEW and implement in Iceberg Nov 28, 2025

findepi reviewed Nov 28, 2025

View reviewed changes

testing/trino-testing/src/main/java/io/trino/testing/BaseConnectorTest.java Outdated Show resolved Hide resolved

piotrrzysko force-pushed the when_stale branch from 8d70622 to 976cbc3 Compare November 28, 2025 11:13

piotrrzysko mentioned this pull request Nov 28, 2025

Support MATERIALIZED VIEW WHEN STALE in analyzer #27502

Draft

findepi changed the title ~~Add WHEN STALE option to CREATE MATERIALIZED VIEW and implement in Iceberg~~ Add WHEN STALE option to CREATE MATERIALIZED VIEW Nov 28, 2025

findepi changed the title ~~Add WHEN STALE option to CREATE MATERIALIZED VIEW~~ Add WHEN STALE syntax for CREATE MATERIALIZED VIEW Nov 28, 2025

findepi approved these changes Nov 28, 2025

View reviewed changes

Praveen2112 approved these changes Nov 28, 2025

View reviewed changes

findepi added no-release-notes This pull request does not require release notes entry and removed syntax-needs-review labels Dec 3, 2025

findepi reviewed Dec 3, 2025

View reviewed changes

piotrrzysko added 2 commits December 3, 2025 14:41

Add WHEN STALE FAIL/INLINE syntax

3079ae6

Add when stale behavior in ConnectorMaterializedViewDefinition

c6b26cc

This is a preparatory change that allows connectors to store information about the WHEN STALE behavior. The option cannot yet be used in practice, as execution will still fail with a NOT_SUPPORTED error.

piotrrzysko force-pushed the when_stale branch from 7eeb933 to d9e7986 Compare December 3, 2025 13:46

findepi force-pushed the when_stale branch from d9e7986 to c6b26cc Compare December 3, 2025 14:06

findepi merged commit cfca263 into trinodb:master Dec 3, 2025
100 checks passed

github-actions bot added this to the 479 milestone Dec 3, 2025

	if (!plannerContext.getMetadata().getConnectorCapabilities(session, catalogHandle).contains(MATERIALIZED_VIEW_WHEN_STALE_BEHAVIOR)) {
	throw semanticException(NOT_SUPPORTED, statement, "Catalog '%s' does not support WHEN STALE", catalogName);
	}

Add WHEN STALE syntax for CREATE MATERIALIZED VIEW #27356

Add WHEN STALE syntax for CREATE MATERIALIZED VIEW #27356

Conversation

piotrrzysko commented Nov 18, 2025 • edited by findepi Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

New syntax

Behavior

Motivation

Additional context and related issues

Release notes

Uh oh!

piotrrzysko Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

findepi Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

Praveen2112 commented Nov 18, 2025

Uh oh!

Uh oh!

Praveen2112 Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

Praveen2112 commented Nov 18, 2025

Uh oh!

martint commented Nov 18, 2025

Uh oh!

martint commented Nov 18, 2025

Uh oh!

piotrrzysko commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

piotrrzysko commented Nov 26, 2025

Uh oh!

findepi commented Nov 26, 2025

Uh oh!

Uh oh!

findepi left a comment

Choose a reason for hiding this comment

Uh oh!

findepi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

findepi left a comment

Choose a reason for hiding this comment

Uh oh!

findepi Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

piotrrzysko Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

Add `WHEN STALE` syntax for `CREATE MATERIALIZED VIEW` #27356

Add `WHEN STALE` syntax for `CREATE MATERIALIZED VIEW` #27356

piotrrzysko commented Nov 18, 2025 •

edited by findepi

Loading

piotrrzysko commented Nov 19, 2025 •

edited

Loading